Distributed Cactus Stacks: Runtime Stack-Sharing Support for Distributed Parallel Programs

Parallel Programming Systems based on the Distributed Shared Memory technique has been promoted as easy to program, natural and equivalent to multiprocessor systems. However, most programmers find this is not the case. The shared memory in DSM systems do not have the same access and sharing semantics as shared memory in real multiprocessor systems (shared memory multiprocessors). We present a scheme, which has been implemented as a part of the Chime Parallel Processing system, that provides, true shared-memory multiprocessor semantics in a distributed system. Programs are written as parallel programs with constructs for parallel forloop and parallel compound statement. A runtime system (middleware) provides the above features using a unique multithreaded architecture. In addition to providing the stack sharing support, the runtime system is able to provide nested parallelism, task synchronization, load balancing and fault tolerance. The software is available at http://milan.eas.asu.edu.